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Event-related  functional  magnetic  resonance  imaging  was  used 
to  measure  blood  oxygenation  level-dependent  responses  in  13 
young  healthy  human  volunteers  during  performance  of  a  prob- 
abilistic reversal-learning  task.  The  task  allowed  the  separate 
investigation  of  the  relearning  of  stimulus-reward  associations 
and  the  reception  of  negative  feedback.  Significant  signal 
change  in  the  right  ventrolateral  prefrontal  cortex  was  demon- 
strated on  trials  when  subjects  stopped  responding  to  the 
previously  relevant  stimulus  and  shifted  responding  to  the 
newly  relevant  stimulus.  Significant  signal  change  in  the  region 
of  the  ventral  striatum  was  also  observed  on  such  reversal 
errors,  from  a  region  of  interest  analysis.  The  ventrolateral 
prefrontal  cortex  and  ventral  striatum  were  not  significantly 


activated  by  the  other,  preceding  reversal  errors,  or  when  sub- 
jects received  negative  feedback  for  correct  responses.  More- 
over, the  response  on  the  final  reversal  error,  before  shifting, 
was  not  modulated  by  the  number  of  preceding  reversal  errors, 
indicating  that  error-related  activity  does  not  simply  accumulate 
in  this  network.  The  signal  change  in  this  ventral  frontostriatal 
circuit  is  therefore  associated  with  reversal  learning  and  Is 
uncontaminated  by  negative  feedback.  Overall,  these  data  con- 
cur with  findings  in  rodents  and  nonhuman  primates  of  reversal- 
learning  deficits  after  damage  to  ventral  frontostriatal  circuitry, 
and  also  support  recent  clinical  findings  using  this  task. 
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Reversal  learning  involves  the  adaptation  of  behavior  according 
to  changes  in  stimulus-reward  contingencies,  a  capacity  relevant 
to  socio-emotional  behavior  (Rolls,  1999).  It  is  exemplified  by 
visual  discrimination  tasks  where  subjects  must  learn  to  respond 
according  to  the  opposite,  previously  irrelevant,  stimulus-reward 
pairing.  Reversal  learning  is  disrupted  after  lesions  of  the  ventral 
prefrontal  cortex  (PFC)  and  ventral  striatum  (VS)  in  nonhuman 
primates  (Divac  et  al.,  1967;  Iversen  and  Mishkin,  1970;  Dias  et 
al.,  1996).  However,  evidence  of  the  same  system  being  involved 
in  reversal  performance  in  humans  is  limited  to  two  studies  in 
patients  with  nonselective  ventral  PFC  damage  (Rolls  et  al.,  1994; 
Rahman  et  al.,  1999).  In  the  current  study,  event-related  func- 
tional magnetic  resonance  imaging  (fMRI)  was  used,  enabling  the 
identification  of  neural  mechanisms  associated  with  reversal 
learning  in  the  intact  human  brain,  with  the  further  aim  of 
dissecting  the  components  and  temporal  dynamics  of  this  process. 

Previous  neuroimaging  studies  have  associated  the  ventral  PFC 
and  VS  with  a  variety  of  functions  related  indirectly  to  reversal 
learning,  including  unconditioned  (Zald  and  Pardo,  1997)  and 
conditioned  (Delgado  et  al.,  2000;  Knutson  et  al.,  2001; 
O'Doherty  et  al.,  2001)  reward  processing  and  low-level  inhibi- 
tory control  (Garavan  et  al.,  1999;  Konishi  et  al.,  1999).  A  recent 
positron  emission  tomography  investigation  using  a  blocked  de- 
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sign  failed  to  identify  blood-flow  changes  in  the  PFC  during 
reversal  learning,  although  changes  were  observed  in  the  ventral 
caudate  nucleus  (Rogers  et  al.,  2000).  However,  in  blocked  de- 
signs a  signal  may  be  attenuated  through  the  averaging  of  activity 
over  an  extended  period.  Methodological  developments  in  fMRI 
have  enabled  the  identification  of  neural  responses  to  single 
events,  and  event-related  fMRI  is  perfectly  suited  to  investigating 
reversal  learning,  where  critical  errors  occur  against  a  background 
of  correct  responses. 

The  aim  of  the  current  study  was  to  explore  the  involvement  of 
ventral  frontostriatal  regions  in  reversal  learning.  The  event- 
related  approach  offers  the  additional  advantage  of  being  able  to 
examine,  for  the  first  time,  the  temporal  dynamics  of  neural 
activity  during  the  reversal  phase  in  humans.  "Final"  reversal 
errors  followed  directly  by  shifting  were  modeled  separately  from 
other  preceding  reversal  errors,  not  directly  leading  to  changes  of 
behavior  (Fig.  1).  The  use  of  a  difficult,  probabilistic  task,  where 
negative  feedback  was  given  to  correct  responses  on  a  minority  of 
trials,  encouraged  perseverative  behavior  after  contingency  rever- 
sals. Separate  investigation  of  probabilistic  errors,  final  reversal 
errors,  and  preceding  reversal  errors  enabled  independent  assess- 
ment of  the  neural  correlates  of  reversal  learning  and  negative 
feedback.  We  predicted  specific  signal  changes  in  a  ventral  fron- 
tostriatal network  during  final  reversal  errors  (directly  leading  to 
changes  in  behavior)  but  not  during  probabilistic  errors  or  pre- 
ceding reversal  errors. 

An  important  rationale  for  the  current  study  was  based  on 
recent  findings  that  performance  on  a  probabilistic  reversal- 
learning  task  was  impaired  by  dopaminergic  medication  in  pa- 
tients with  mild  Parkinson's  disease  (Swainson  et  al.,  2000;  Cools 
et  al.,  2001).  This  detrimental  eifect  of  dopamine  was  hypothe- 
sized to  be  a  result  of  "overdosing"  a  ventral  frontostriatal  circuit 
(Swainson  et  al.,  2000;  Cools  et  al.,  2001),  given  neuroanatomical 
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Figure  1.  The  probabilistic  reversal-learning  task.  An  example  of  several 
consecutive  trials  in  the  probabilistic  reversal-learning  task  is  shown 
(running  from  bottom  to  top).  On  each  trial,  subjects  are  presented  with 
two  abstract  visual  patterns.  Using  trial-and-error  feedback,  subjects  must 
discover  which  of  the  two  patterns  is  correct  (here  indicated  by  a  small 
arrowhead  on  top  of  the  pattern  to  improve  legibility).  Feedback  (a  green 
smiley  face  or  red  sad  face)  is  presented  as  soon  as  the  subject  has  chosen 
one  of  the  patterns  by  a  left  or  right  button  press. 

evidence  for  a  relative  preservation  of  the  VS  in  the  early  stages 
of  the  disease  (Kish  et  al.,  1988).  Confirmation  of  ventral  fron- 
tostriatal  involvement  in  reversal  learning  would  considerably 
strengthen  this  "overdose"  hypothesis. 

MATERIALS  AND  METHODS 

Subjects.  Fourteen  right-handed,  young,  healthy  volunteers  participated 
in  this  study.  One  subject  was  unable  to  perform  satisfactorily  on  the  task 
and  was  therefore  excluded  from  the  analysis.  All  remaining  13  subjects 
(5  males,  8  females;  mean  age,  25.9;  SD,  3.82;  range,  22-37)  gave 
informed  consent,  which  was  approved  by  the  local  Research  Ethics 
Committee. 

Experimental  design.  Each  subject  was  scanned  performing  the  behav- 
ioral task  in  three  successive  9  min  sessions.  Before  entering  the  scanner, 
subjects  performed  a  30  trial  training  session.  This  was  a  simple  proba- 
bilistic discrimination  task  (i.e.,  without  reversal  stages)  designed  to 
introduce  the  subject  to  the  concept  of  a  probabilistic  error  without  the 
need  to  reverse  responding.  On  each  go,  the  same  two  patterns  were 
presented.  One  of  the  patterns  was  correct  and  the  other  pattern  was 
wrong,  and  subjects  had  to  choose  the  correct  pattern  on  each  go.  During 


the  task,  the  rule  changed  intermittently  so  that  the  other  pattern  was 
usually  correct.  Subjects  were  instructed  to  only  start  choosing  the  other 
pattern  when  they  were  sure  that  the  rule  had  changed. 

The  task  was  programmed  in  Microsoft  (Seattle,  WA)  Visual  Basic  6.0 
and  stimuli  were  presented  on  a  computer  display  projected  onto  a 
mirror  in  the  MRI  scanner.  Different  stimuli  were  used  in  each  of  the 
three  task  blocks  (and  training  stage),  and  the  order  of  presentation  of 
the  blocks  was  counter-balanced  across  subjects.  Each  block  consisted  of 
10  discrimination  stages,  and  therefore,  9  reversal  stages.  Reversal  of  the 
stimulus-reward  contingency  occurred  after  between  10  and  15  correct 
responses  (including  probabilistic  errors).  The  number  of  probabilistic 
errors  between  each  reversal  varied  from  0  to  4.  To  prevent  subjects  from 
adopting  a  strategy  such  as  always  reversing  after  two  consecutive  errors, 
probabilistic  negative  feedback  was  given  on  two  consecutive  trials  once 
during  each  task  block.  Each  block  lasted  —8.5  min  depending  on  level  of 
performance.  The  two  stimuli  in  each  block  were  abstract  colored  pat- 
terns presented  simultaneously  in  the  left  and  right  visual  fields  (location 
randomized)  (Fig.  1).  Responses  were  made  using  the  left  or  right  button 
on  a  button  box  positioned  on  the  stomach  of  the  subject.  On  each 
individual  trial,  the  stimuh  were  presented  for  2000  msec  within  which 
the  response  had  to  be  made  (or  else  a  "too  late"  message  was  presented). 
Feedback,  consisting  of  a  green  smiley  face  for  correct  responses  or  a  red 
sad  face  for  incorrect  responses,  was  presented  immediately  after  the 
response  (Fig.  1).  The  feedback  faces  were  presented  centrally,  between 
the  2  stimuli,  for  500  msec  during  which  the  stimuli  also  remained  on  the 
screen.  After  feedback,  the  stimuli  were  removed  and  the  face  was 
replaced  by  a  fixation  cross  for  a  variable  interval  so  that  the  overall 
interstimulus  interval  was  3253  msec,  enabling  precise  desynchronization 
from  the  repetition  time  (TR)  (of  3000  msec)  and  sufficient  sampling 
across  the  hemodynamic  response  function. 

Imaging  acquisition.  Imaging  data  were  collected  using  a  Bruker  Med- 
spec  scanner  (S300;  Bruker,  Ettlingen,  Germany)  operating  at  3  tesla.  A 
total  of  180  T2*-weighted  echo-planar  images  (EPIs),  depicting  blood 
oxygenation  level-dependent  contrast,  were  acquired  in  each  session 
(TR,  3  sec;  echo  time,  27  msec).  A  total  of  21  slices  (each  of  4  mm 
thickness;  interslice  gap,  5  mm;  matrix  size,  64  X  64;  bandwidth,  100 
kHz;  axial  oblique  acquisition  orientation)  per  image  were  acquired.  The 
first  seven  EPIs  in  each  session  were  discarded  to  avoid  Tl  equilibrium 
effects.  We  were  unable  to  collect  data  from  the  orbitofrontal  and 
ventromedial  PFC  because  of  susceptibility  artifacts  in  nasal  sinuses 
leading  to  signal  dropout. 

Imaging  analysis.  Data  analysis  was  performed  using  SPM  99  (Statis- 
tical Parametric  Mapping;  Wellcome  Department  of  Cognitive  Neurol- 
ogy, London,  UK).  Preprocessing  procedures  included  slice  acquisition 
time  correction,  reorientation,  within-subject  realignment,  geometric  un- 
distortion  using  fieldmaps  (Cusack  et  al.,  2001),  spatial  normalization 
using  EPI  masking  (to  exclude  areas  susceptible  to  signal  dropout  from 
nonlinear  warping)  to  the  standard  Montreal  Neurological  Institute  EPI 
template,  and  spatial  smoothing  using  a  Gaussian  kernel  (8  mm  full- 
width  at  half-maximum).  Time  series  were  high-pass  filtered. 

A  canonical  hemodynamic  response  function  was  used  as  a  covariate  in 
a  general  linear  model  and  a  parameter  estimate  was  generated  for  each 
voxel  for  each  event  type.  The  parameter  estimate,  derived  from  the 
mean  least  squares  fit  of  the  model  to  the  data,  reflects  the  strength  of 
covariance  between  the  data  and  the  canonical  response  function  for  a 
given  condition.  Individuals'  contrast  images,  derived  from  pair-wise 
contrasts  between  parameter  estimates  for  different  events,  were  taken  to 
a  second-level  group  analysis  in  which  t  values  were  calculated  for  each 
voxel  treating  intersubject  variability  as  a  random  effect.  The  t  values 
were  transformed  to  unit  normal  Z  distribution  to  create  a  statistical 
parametric  map  for  each  of  the  planned  contrasts  (described  below). 

The  hemodynamic  response  function  was  modeled  to  the  onset  of  the 
responses,  which  co-occurred  with  the  presentation  of  the  feedback.  The 
following  events  were  modeled  (Fig.  1):  (1)  correct  responses,  co- 
occurring  with  positive  feedback,  as  a  baseline;  (2)  probabilistic  errors, 
on  which  negative  feedback  was  given  to  correct  responses  (trials  on 
which  subjects  reversed  after  a  probabilistic  error  were  not  included  in 
the  model);  (3)  final  reversal  errors,  resulting  in  the  subject  shifting  their 
responding;  and  (4)  the  other  preceding  reversal  errors,  following  a 
contingency  reversal  but  preceding  the  final  reversal  errors.  The  final 
reversal  errors  (co-occurring  with  the  last  negative  feedback)  were  cho- 
sen as  critical  events  of  interest  (i.e.,  reflecting  reversal  learning)  because 
activation  of  a  reversal  network  was  assumed  to  follow  this  last  negative 
feedback.  Error  trials  that  could  not  be  classified  as  probabilistic  errors  or 
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reversal  errors  (so-called  "spontaneous"  errors)  were  not  included  in  the 
model. 

The  following  contrasts  were  assessed:  (1)  final  reversal  errors  minus 
correct  responses,  (2)  other  preceding  reversal  errors  minus  correct 
responses,  (3)  probabilistic  errors  minus  correct  responses,  (4)  final 
reversal  errors  minus  other  preceding  reversal  errors,  and  (5)  final 
reversal  errors  minus  probabilistic  errors.  In  addition,  we  assessed 
whether  the  final  reversal  errors  were  parametrically  modulated  by  the 
number  of  reversal  errors  directly  preceding  them.  We  predicted  signif- 
icant signal  change  in  the  ventral  frontostriatal  brain  regions  in  contrasts 
1,  4,  and  5  but  not  in  contrasts  2  or  3.  All  contrasts  were  initially 
thresholded  at  p  <  0.05,  corrected  for  multiple  comparisons.  Strong 
predictions  about  the  involvement  of  the  VS  in  reversal  learning  (see  the 
introductory  remarks)  justified  application  of  small  volume  corrections 
using  a  sphere  around  the  VS.  This  a  priori  defined  region  of  interest 
(ROI)  was  a  sphere  centered  onx,  y,  z  =  +/—10,  8,  -4  with  a  radius  of 
8  mm  (i.e.,  the  smoothing  kernel).  These  coordinates  represent  the  center 
of  the  nucleus  accumbens  as  defined  using  the  Talairach  atlas,  and  are  very 
close  to  ventral  striatal  foci  specified  in  previous  fMRI  studies  (Delgado  et 
al.,  2000;  Breiter  et  al.,  2001;  Knutson  et  al.,  2001).  These  coordinates  were 
unaltered  by  conversion  from  Talairach  to  Montreal  Neurological  Institute 
(MNI)  space  using  an  algorithm  by  M.  Brett  (Medical  Research  Council 
Cognition  and  Brain  Sciences  Unit,  Cambridge,  UK)  (available  at 
www.mrc-cbu.cam.ac.uk/imaging). 

Finally,  ROI  analyses  were  performed  on  the  less  powerful  contrasts  4 
and  5,  which  did  not  include  basehne  correct  responses  but  were  pre- 
dicted to  generate  ventrolateral  PFC  signal  change.  A  region  in  the 
ventrolateral  PFC  was  defined  on  the  basis  of  previous  functional  neu- 
roimaging  studies.  Specifically,  the  ventral  PFC  has  been  activated  re- 
peatedly in  studies  of  working  memory  (Owen,  1997),  and  on  the  basis  of 
these  studies  coordinates  have  been  reported  that  define  the  approximate 
functional  extent  of  this  neuroanatomical  region  (stereotaxic  coordinates 
X  =  +1-26  tox  =  +/-50,>'  =  +16  toy  =  +2A,  and  z  =  -9  to  z  =  8) 
(Owen  et  al.,  1999).  The  only  fMRI  study  specifically  looking  at  reversal 
learning  did  not  scan  below  the  anterior  commissure -posterior  commis- 
sure (AC-PC)  axis  (Nagahama  et  al.,  2001).  The  above-described  sta- 
tistical model  was  then  reapplied  to  the  average  signal  within  the  ROI, 
using  the  MarsBar  tool  (M.  Brett,  personal  communication;  see 
www.mrc-cbu.cam.ac.uk/imaging/marsbar.html). 

RESULTS 
Behavioral  data 

All  13  subjects  included  in  the  analysis  performed  well  on  the 
task.  On  average,  subjects  made  2.6  (SD,  0.51)  perseverative 
reversal  errors  after  reversal  of  stimulus-reward  contingencies. 
Over  the  task  as  a  whole,  subjects  made  on  average  (SD)  320.7 
(4.4)  correct  responses,  48.4  (2.7)  probabilistic  errors,  43.4  (13.9) 
preceding  reversal  errors,  and  27  (0)  final  reversal  errors. 

Imaging  data 

Significant  effects  observed  in  whole -brain  analyses  are  displayed 
in  Table  1. 

Comparison  of  the  final  reversal  errors  with  the  baseline  cor- 
rect responses  (contrast  1;  see  Materials  and  Methods)  revealed 
significant  signal  change  in  the  right  ventrolateral  PFC  (Table  1 
and  Fig.  2).  The  effect  in  the  left  ventrolateral  PFC  was  present 
but  did  not  reach  significance  (coordinates  x,  _y,  z  =  —32,  24,  —4; 
T  =  6.8).  Other  significant  effects  in  this  contrast  were  observed 
in  medial  frontal  cortex  (Brodmann  area  8)  and  right  parietal 
cortex  (Table  1).  Small  volume  corrections,  restricting  the  search 
volume  to  a  sphere  around  the  ventral  striatum  (see  Materials  and 
Methods)  revealed  significant  signal  changes  in  that  region  bilat- 
erally (coordinates  X,  >;,  z  =  —10,  2,  —2;  T  =  6.3;  p  <  0.003;  and 
coordinates  X,  y,  z  =  14,  2,  —6;  T  =  4.3;  p  =  0.03). 

An  ROI  analysis,  restricting  the  search  volume  to  an  indepen- 
dently defined  area  in  the  ventrolateral  PFC  (see  Materials  and 
Methods),  revealed  that  signal  change  in  the  right  ROI  during  the 
final  reversal  error  was  also  significantly  greater  than  that  ob- 
served during  the  preceding  reversal  errors  (contrast  A;  T  =  2.79; 


Table  1.  Significant  effects  observed  in  wliole-brain 

1  analyses 

Coordinates 

V 

Brain  area 

X,  y,  z 

r 

(corrected) 

Final  reversal  errors  minus 

correct  responses 

Right  ventrolateral  PFC 

38,  24,  -2 

14.6 

<0.0001 

Right  medial  frontal  cortex 

(Brodmann's  area  8) 

8,  32,  52 

10.7 

0.014 

Right  parietal  cortex 

42,  -42,  40 

10.5 

0.017 

Preceding  reversal  errors 

minus  correct  responses 

Right  premotor  cortex 

(Brodmann's  area  6) 

10,  10,  52 

10.52 

0.017 

Coordinates  are  in  standard  MNI  spaced  as  given  by  Statistical  Parametric  Mapping 
99.  Conversion  of  coordinates  from  MNI  space  to  Talairach  space  did  not  lead  to 
localization  changes. 

p  =  0.016)  and  during  the  probabilistic  errors  (contrast  5;  T  = 
2.84;  p  =  0.014). 

Significant  effects  in  the  ventrolateral  PFC  and  VS  were  absent 
when  the  other  preceding  reversal  errors  were  contrasted  with  the 
baseline  correct  responses  (contrast  2)  and  when  the  probabilistic 
errors  were  contrasted  with  the  baseline  correct  responses  (con- 
trast 3).  Moreover,  there  was  no  significant  parametric  effect  at 
the  final  reversal  errors  as  a  function  of  the  number  of  preceding 
reversal  errors,  even  when  the  search  volume  was  restricted  to  the 
ventrolateral  PFC  using  an  ROI  analysis. 

DISCUSSION 

The  present  results  demonstrate  recruitment  of  a  ventral  fronto- 
striatal system  in  a  task  of  probabilistic  reversal  learning.  Detailed 
analyses  showed  that  this  significant  signal  change,  observed  in 
the  right  ventrolateral  PFC  and  in  the  region  of  the  VS,  occurred 
specifically  during  the  final  reversal  error,  at  which  point  subjects 
stopped  responding  to  a  previously  relevant  pattern  and  reversed 
responding  to  a  newly  relevant  pattern. 

These  data  are  consistent  with  our  predictions  and  concur  with 
primate  and  rodent  lesion  studies  showing  that  damage  to  the 
ventral  PFC  (Iversen  and  Mishkin,  1970;  Jones  and  Mishkin, 
1972;  Dias  et  al,  1996)  and  VS  (Divac  et  al.,  1967;  Taghzouti  et 
al.,  1985;  Annett  et  al.,  1989;  Stern  and  Passingham,  1995)  dis- 
rupts reversal  learning.  For  example.  Stern  and  Passingham 
(1995)  have  shown  that  lesions  of  the  nucleus  accumbens  in 
monkeys  lead  to  deficits  on  tasks  of  spatial  (but  not  object  or 
motor)  reversal  learning,  while  leaving  acquisition  performance 
intact.  In  rats,  dopamine  (6-OHDA)  and  ibotenic  acid  lesions  of 
the  nucleus  accumbens  have  led  to  both  acquisition  and  reversal- 
learning  impairments  in  a  spatial  T-maze  and  a  Morris  water 
maze  (Taghzouti  et  al.,  1985;  Annett  et  al.,  1989),  suggesting  a 
role  for  the  nucleus  accumbens  in  the  relearning  of  new  location- 
reward  associations,  rather  than  in  stopping  old  responses  (An- 
nett et  al.,  1989).  The  current  study  indicates  that  the  VS,  at  least 
in  humans,  is  also  implicated  in  the  reversal  or  the  relearning  of 
object-reward  associations.  In  contrast,  human  brain-imaging 
studies  have  emphasized  a  role  for  the  right  lateral  ventral  PFC  in 
behavioral  inhibition  (or  stopping)  using,  for  example,  go-nogo 
tasks  (Garavan  et  al.,  1999;  Konishi  et  al.,  1999).  It  is  possible  that 
the  observed  signal  change  in  the  ventrolateral  PFC,  also  lateral- 
ized  to  the  right  hemisphere,  reflects  behavioral  inhibition, 
whereas  the  signal  change  in  the  VS  reflects  the  learning  of  new 
associations.  However,  our  study  was  not  designed  to  functionally 
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Figure  2.  Signal  changes  in  ventral  frontostriatal  circuitry  during  the  critical  final  reversal  errors.  Signal  changes  in  the  bilateral  ventrolateral  PFC  and 
ventral  striatum,  identified  by  the  contrast,  final  reversal  errors  minus  correct  responses,  are  superimposed  on  the  MNI  template  brain  (individual  brain 
considered  most  typical  of  the  305  brains  used  to  define  the  MNI  standard).  See  Results  and  Table  1  for  statistical  values.  In  all  three  axial  slices,  the 
z-coordinate  represents  the  position  of  the  slice  relative  to  the  anterior  commissure -posterior  commissure  axis  (dorsal,  positive). 


dissociate  learning  from  stopping;  this  could  be  addressed  in 
future  event-related  brain-imaging  studies. 

The  use  of  an  event-related  technique  enabled  the  separate 
investigation  of  distinct  error  trial  types  that  loaded  differentially 
on  reversal  shifting  and  simple  negative  feedback.  The  signal 
change  in  the  ventrolateral  PFC  and  VS  was  significantly  greater 
at  the  final  reversal  error  compared  with  baseline  correct  re- 
sponses. A  region  of  interest  analysis  revealed  that  signal  change 
in  the  right  ventrolateral  PFC  was  also  significantly  greater  during 
the  final  reversal  error  than  during  all  other  error  trial  types. 
Moreover,  the  absence  of  a  parametric  effect  suggests  that  the 
effects  at  the  final  reversal  error  were  not  modulated  by  the 
number  of  preceding  reversal  errors.  This  indicates  that  the  focus 
was  not  the  result  of  gradual  accumulation  of  activity  caused  by 
the  preceding  errors,  although  given  power  considerations  such 
conclusions  cannot  be  considered  to  be  definitive.  Finally,  these 
areas  were  not  significantly  activated  during  the  preceding  rever- 
sal errors  or  probabilistic  errors  when  compared  with  baseline 
correct  responses.  Although  it  is  not  possible  conceptually  to 
doubly  dissociate  reversal  learning  from  negative  feedback,  these 
results  suggest  that  the  effect  during  the  final  reversal  error  in  the 
ventrolateral  PFC  is  primarily  attributable  to  reversal  learning 
and  cannot  be  explained  by  an  effect  of  negative  feedback. 

These  findings  are  broadly  consistent  with  those  observed  in  a 
recent  event -related  fMRI  study  of  high-level  attentional  set  shift- 
ing (Monchi  et  al.,  2001).  Monchi  et  al.  (2001)  demonstrated 
signal  change  in  an  area  in  the  ventrolateral  PFC,  at  coordinates 
very  similar  to  the  current  focus,  in  response  to  negative  feed- 
back, signaling  a  shift  of  set.  However,  in  that  study  it  was  not 
possible  to  isolate  the  shifting  component  from  the  negative- 
feedback  component.  Thus,  the  current  study  extends  their  re- 
sults by  showing  that  the  effect  in  the  ventrolateral  PFC  is  uncon- 
taminated  by  negative  feedback.  The  present  data  also  indicate 
that  shifting  of  lower-level  stimulus-reward  associations,  as  op- 
posed to  shifting  of  a  higher-level  attentional  set,  is  sufficient  to 
activate  the  ventrolateral  PFC. 

Two  additional  neuroimaging  studies  have  used  reversal- 
learning  tasks  with  event-related  fMRI.  Nagahama  et  al.  (2001) 


did  not  scan  brain  areas  below  the  horizontal  plane  through  the 
AC-PC  axis  (Talairach  coordinate  z  =  0).  This  precluded  con- 
clusions about  the  role  of  the  VS  and  ventral  PFC  in  reversal 
learning.  Instead,  they  emphasized  a  role  for  a  (more  dorsal) 
posteroventral  PFC  area  in  shifting,  which  was  not  replicated  in 
the  current  study.  However,  the  focus  in  this  posteroventral  area 
was  later  clarified  by  Monchi  et  al.  (2001)  to  be  nonspecifically 
activated  [i.e.,  to  respond  during  both  negative  feedback  (trigger- 
ing set  shifting)  and  positive  feedback  (triggering  set  mainte- 
nance)]. The  relevant  contrasts  (subtracting  positive -feedback 
trials  from  negative -feedback  trials)  in  the  current  study  were  not 
designed  to  address  this  question  of  nonspecific  feedback.  A 
second  event-related  fMRI  study,  using  a  probabilistic  reversal- 
learning  paradigm  to  assess  orbitofrontal  neural  responses  to 
reward  and  punishment  (O'Doherty  et  al.,  2001),  revealed  signal 
change  in  the  right  ventrolateral  PFC  during  reception  of  nega- 
tive feedback.  This  signal  change  was  interpreted  to  reflect  pun- 
ishment. However,  it  was  not  possible  in  that  study  to  exclude  the 
contribution  of  reversal  learning.  In  fact,  our  results  show  that  this 
ventrolateral  PFC  effect  more  likely  reflects  reversal  learning.  In 
addition,  O'Doherty  et  al.  (2001)  observed  signal  change  in  the 
medial  orbitofrontal  cortex  that  correlated  with  the  magnitude  of 
reward,  and  observed  signal  change  in  the  lateral  orbitofrontal 
cortex  that  correlated  with  the  magnitude  of  punishment.  Be- 
cause of  susceptibility  artifacts,  we  were  unable  to  image  these 
latter  brain  regions  and  are  therefore  unable  to  draw  conclusions 
on  orbitofrontal  effects  during  trials  on  which  negative  feedback 
was  received  without  consequences  for  behavior  (as  on  probabi- 
listic errors  and  preceding  reversal  errors). 

The  orbitofrontal  cortex  is  connected  to  the  nucleus  accumbens 
of  the  ventral  striatum  in  a  segregated  frontostriatal  "loop"  (Al- 
exander et  al.,  1986;  Groenewegen  et  al.,  1997).  Functional  evi- 
dence implicates  the  orbitofrontal  cortex,  in  interaction  with  the 
amygdala,  in  unconditioned  (and  conditioned)  reward  processing 
(Delgado  et  al,  2000;  Breiter  et  al.,  2001;  Knutson  et  al,  2001; 
O'Doherty  et  al.,  2001).  In  contrast,  the  ventrolateral  PFC  is 
connected  to  the  ventral  putamen,  a  structure  implicated  in  motor 
function.  The  integrative  role  of  the  ventral  striatum  has  been 
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suggested  to  be  the  funnelling  of  motivational  information  from 
the  "limbic"  system  to  the  motor  system  (Mogenson,  1987), 
thereby  mediating  the  effects  of  stimulus-reward  mechanisms  on 
goal-directed  behavior  (Robbins  et  al.,  1989;  Schultz  et  al.,  1992). 
Thus,  whereas  the  orbitofrontal  cortex  may  be  important  for  the 
"low-level"  representation  of  reward  or  punishment  values 
(O'Doherty  et  al.,  2001),  the  more  lateral  PFC  may  play  a  role  in 
the  adaptation  of  behavior  in  response  to  changes  in  such  reward 
or  punishment  values.  Our  finding  that  the  ventrolateral  PFC  is 
critically  involved  in  reversal  learning,  uncontaminated  by  the 
reception  of  negative  feedback  per  se,  is  consistent  with  this 
proposed  hierarchy  within  corticostriatal  systems. 

Finally,  our  results  provide  a  clear  interpretation  of  recent  data 
demonstrating  that  administration  of  dopaminergic  medication  to 
patients  with  mild  Parkinson's  disease  has  a  detrimental  effect  on 
probabilistic  reversal  learning  (Swainson  et  al.,  2000;  Cools  et  al., 
2001).  Recent  studies  have  shown  that,  in  early  Parkinson's  dis- 
ease, dopamine  depletion  is  restricted  to  the  putamen  and  the 
dorsal  caudate  nucleus,  only  later  progressing  to  more  ventral 
parts  of  the  striatum  and  the  mesocorticolimbic  system  (Kish  et 
al.,  1988;  Agid  et  al.,  1993).  It  was  hypothesized  that  administra- 
tion of  L-3,4-dihydroxyphenylalanine  (l-DOPA)  doses,  necessary 
to  remediate  the  dopamine  depletion  in  the  dorsal  striatum  and 
its  connections  to  the  dorsolateral  PFC,  may  detrimentally  over- 
dose relatively  intact  brain  regions,  such  as  the  VS  and  its  con- 
nections to  the  ventral  PFC  (Gotham  et  al.,  1988;  Cools  et  al., 
2001).  The  current  data,  showing  involvement  of  the  ventral  PFC 
and  VS  in  probabilistic  reversal  learning,  considerably  strengthen 
the  possibility  that  dopaminergic  agents  can  indeed  overdose  a 
relatively  intact  ventral  frontostriatal  system.  This  work  highlights 
the  potential  of  combining  pharmacological  and  functional  imag- 
ing approaches  to  understand  the  underlying  neural  substrates  of 
cognitive  functions  in  both  basic  and  clinical  settings. 
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